Displaying In-Memory Video using OpenGL

By Michael Portuesi and Nelson Bolyard. Special thanks to Robert Tray for some of the OpenGL information.

Introduction

A common question asked by beginning VL programmers is, "How do I display video using OpenGL?"

It is not surprising that people ask this question. There are few code examples in existence which demonstrate the use of VL and OpenGL in the same program. To compound matters, the subject is also not mentioned in the Iris Media Libraries Programming Guide, which was written around IRIS GL-based code examples.

Setting up OpenGL

The biggest problem that one encounters when trying to set up video display through OpenGL is that the video display is slow - much slower, in fact, than that produced by the IRIS GL-based code samples.

But video display through OpenGL does not have to be slow, if you perform the proper setup work. Here's the checklist:

• When you create an openGL window, you will do one of two things:
  1. If you are a pure X programmer, you call glxGetConfig() and glxChooseVisual() to create an X window suitable for OpenGL rendering.
     
     
  2. if you are a Motif programmer, you call GLwCreateMDrawingArea() to create an OpenGL widget.
     
     Either way, you eventually call glXCreateContext() to associate an OpenGL context with the window. The call might look like this:
     
     context = glXCreateContext(XtDisplay(widget), vi, 0, GL_TRUE);
     
     It is very important the last parameter is GL_TRUE. This specifies that pixel rendering should be done directly to the hardware, rather than through the X server.
     
     
• You need to turn off some OpenGL features that can slow down pixel transfers. Here is a piece of code to do it:

/*
 * The original version of this code was developed by Allen Akin
 * (akin@sgi.com).  It has been modified for this example.
 */
void setupGL()
{
    /*
     * Is there a 24-bit visual? if not, we want to dither RGB.
     */
    Boolean doDither = TRUE;
    Display * dpy    = XtDisplay(getDeviceWidget());
    XVisualInfo	vinfo;
    XVisualInfo *viList;
    int nitems;

    vinfo.depth = 24;
    viList = XGetVisualInfo(dpy, VisualDepthMask, &vinfo, &nitems);
    if (viList) {
	XFree(viList);
	doDither = FALSE;
    }

    /*
     * Disable stuff that's likely to slow down glDrawPixels.
     * (Omit as much of this as possible, when you know in advance
     * that the OpenGL state will already be set correctly.)
     */
    glDisable(GL_ALPHA_TEST);
    glDisable(GL_BLEND);
    glDisable(GL_DEPTH_TEST);
    if (!doDither) {
	glDisable(GL_DITHER);
    }
    glDisable(GL_FOG);
    glDisable(GL_LIGHTING);
    glDisable(GL_LOGIC_OP);
    glDisable(GL_STENCIL_TEST);
    glDisable(GL_TEXTURE_1D);
    glDisable(GL_TEXTURE_2D);
    glPixelTransferi(GL_MAP_COLOR, GL_FALSE);
    glPixelTransferi(GL_RED_SCALE, 1);
    glPixelTransferi(GL_RED_BIAS, 0);
    glPixelTransferi(GL_GREEN_SCALE, 1);
    glPixelTransferi(GL_GREEN_BIAS, 0);
    glPixelTransferi(GL_BLUE_SCALE, 1);
    glPixelTransferi(GL_BLUE_BIAS, 0);
    glPixelTransferi(GL_ALPHA_SCALE, 1);
    glPixelTransferi(GL_ALPHA_BIAS, 0);

    /*
     * Disable extensions that could slow down glDrawPixels.
     */
    const GLubyte* extString = glGetString(GL_EXTENSIONS);

    if (extString != NULL) {
       if (strstr((char*) extString, "GL_EXT_convolution") != NULL) {
           glDisable(GL_CONVOLUTION_1D_EXT);
           glDisable(GL_CONVOLUTION_2D_EXT);
           glDisable(GL_SEPARABLE_2D_EXT);
       }
       if (strstr((char*) extString, "GL_EXT_histogram") != NULL) {
           glDisable(GL_HISTOGRAM_EXT);
           glDisable(GL_MINMAX_EXT);
       }
       if (strstr((char*) extString, "GL_EXT_texture3D") != NULL) {
           glDisable(GL_TEXTURE_3D_EXT);
       }
    }
}

• Video that you get from VL is arranged in top-to-bottom orientation, whereas openGL works in bottom-to-top orientation. So, before you call glDrawPixels(), do this:
  
  glRasterPos2i(originX, originY);
  glPixelZoom(1.0, -1.0);
  
  originX and originY should be the upper left corner of the region you want to draw, in openGL coordinates (origin in lower left). The glPixelZoom() call tells OpenGL to flip the pixels on the way to the display. On the Indy, this is a highly optimized operation.
  
  
• Draw the pixels using SGI's ABGR extension to OpenGL. On the Indy at least, this means the pixels from VL will go directly to the screen. Like this:
  
  glDrawPixels(sizeW, sizeH, GL_ABGR_EXT, GL_UNSIGNED_BYTE, dataPtr);
  
  
• If you do not plan to do anything with the pixels from VL except display them on the screen, there is a VL optimization you can perform to maximize video throughput. Perform the following call after vlCreateBuffer() and before vlRegisterBuffer():
  
  vlBufferAdvise(buf, VL_BUFFER_ADVISE_NOACCESS);

These steps are sufficient to ensure that video displays quickly. But there are further subtleties you must pay attention to in order to insure reliable transfers. These fine points, such as setting up VL events, and the proper way to respond to certain VL events, is discussed in detail elsewhere in this guide.

Displaying 8-bit Video using OpenGL

The above discussion assumes that you are displaying 24-bit video (VL packing VL_PACKING_RGB_8) to the screen. Some devices, such as VINO, support 8-bit video-to-memory transfers (VL packing VL_PACKING_RGB_332_P).

The following method for displaying an 8-bit video stream using OpenGL is provided by Nelson Bolyard:

The trick for efficiently displaying 8-bit VINO BGR233 images is quite involved, and entirely undocumented, as far as I know. Credit for the code below goes to Terry Crane (tcrane@headcase.esd.sgi.com). It involves using OpenGL's built-in "pixel mapping", which is another form of color mapping, that is separate and distinct from and in addition to the X-server's color mapping. Indy and Indigo2's XL graphics have hardware acceleration for this "pixel mapping" that translates dithered BGR233 into RGBA.

To use it, you first setup OpenGL's state machine with this code:

static void
FastUByteCItoRGBAPixelMap()
{
    GLint i;
    GLfloat constantAlpha = 1.0;
    GLfloat map[256];

    glPixelTransferf(GL_ALPHA_SCALE, 0.0);
    glPixelTransferf(GL_ALPHA_BIAS,  1.0);
    glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

    /* define accelerated bgr233 to RGBA pixelmaps.  */
    for(i=0; i<256; i++)
        map[i] = (i&0x7)/7.0;
    glPixelMapfv(GL_PIXEL_MAP_I_TO_R, 256, map);
    for(i=0; i<256; i++)
        map[i] = ((i&0x38)>>3)/7.0;
    glPixelMapfv(GL_PIXEL_MAP_I_TO_G, 256, map);
    for(i=0; i<256; i++)
        map[i] = ((i&0xc0)>>6)/3.0;
    glPixelMapfv(GL_PIXEL_MAP_I_TO_B, 256, map);

    glPixelMapfv(GL_PIXEL_MAP_I_TO_A, 1, &constantAlpha);

    glPixelTransferi(GL_INDEX_SHIFT, 0);
    glPixelTransferi(GL_INDEX_OFFSET, 0);
    glPixelTransferi(GL_MAP_COLOR, GL_TRUE);
    glDisable(GL_DITHER);
}

and then you invoke glDrawPixels with the "format" GL_COLOR_INDEX.